Skip to content

Temperature-departure lit review: rag-build + 10 papers + cite map (1/3)#60

Merged
NewGraphEnvironment merged 8 commits into
mainfrom
58-temperature-lit-review
May 5, 2026
Merged

Temperature-departure lit review: rag-build + 10 papers + cite map (1/3)#60
NewGraphEnvironment merged 8 commits into
mainfrom
58-temperature-lit-review

Conversation

@NewGraphEnvironment
Copy link
Copy Markdown
Owner

Summary

  • Issue 1 of the climate-departure 3-split lit reviews (temperature; precip+drying and interpretation framing follow as separate issues). Mirrors the Snowpack-departure methodology lit review: Zotero + ragnar + cited vignette #53/Snowpack-departure methodology lit review: rag-build + 11 papers + citation map #54/v0.1.7 snowpack-lit-review pattern verbatim — produces a citation backbone that the downstream vignette branch consumes via [@key] markers.
  • 10 new peer-reviewed papers added to the NewGraphEnvironment/climate Zotero collection (key 8MH9LCC9) with PDFs attached: Karl 1993 (DTR foundational), Easterling 1997 / Vose 2005 (DTR globe trends), Vincent 2018 (Canada/BC trends), Pepin 2015 / Rangwala & Miller 2012 (elevation-dependent warming), Wang 2012 (ClimateWNA BC downscaling), Mantua 2010 (PNW salmon climate-stress bridge), Eaton & Scheller 1996 / Richter & Kolmes 2005 (salmonid thermal envelope). Plus 7 reuse-relevant existing climate-collection items + 2 cross-rag references from the snow methodology rag.
  • Local ragnar DuckDB at data/rag/temp_methodology.duckdb (677 chunks, 10 sources, ~28 s ingest via Ollama nomic-embed-text), built by the new scripts/rag_temp_methodology_build.R. 24-query mining via scripts/rag_temp_methodology_query.R produced planning/active/temp_methodology_quotes.md (637 lines), synthesized into findings.md with methodology-quotes-by-topic, cross-cutting methodology, deviations, and an 18-row "cite this for that" map (BBT-auto-derived keys, ready for downstream [@key] insertion).
  • Phase 0 prep: renamed the existing rag_build_*.R / rag_query_*.R scripts to rag_*_build.R / rag_*_query.R (noun_verb to match the cd_* function convention). 3 file renames + docstring/CLAUDE.md/archive-README updates.
  • Sidecar issues filed: soul#43 documents the BBT-auto-derived citation key convention so future /lit-search + /zotero-api runs avoid manual Citation Key: overrides; cd#59 is the BEC zone-shift lit-review tracker (placeholder, post-3-split).

Headline finding: cd_trend()'s raw Mann-Kendall + Theil-Sen aligns with Vincent et al. 2018 (Canadian temperature trend methodology — Sen slope + Kendall's τ), modulo their iterative lag-1 AC handling. Yue & Wang 2002 (cross-rag) supports our raw form for 76-year strong-trend series.

Relates to NewGraphEnvironment/sred-2025-2026#23.

Fixes #58.

Test plan

  • devtools::test() clean on 58-temperature-lit-review (verified: 166 PASS, 0 FAIL)
  • lintr::lint('scripts/rag_temp_methodology_build.R') + _query.R → 0 lints (verified)
  • Rscript scripts/rag_temp_methodology_build.R reproduces the store (10 sources, ~677 chunks) given the local PDF cache populated from Zotero
  • Rscript scripts/rag_temp_methodology_query.R regenerates planning/active/temp_methodology_quotes.md
  • All 10 BBT citation keys in findings.md Phase 2 table resolve to items in NewGraphEnvironment/climate (verified via local sqlite)
  • PWF checkboxes in planning/active/task_plan.md match landed work
  • No vignette edits on this branch (lit review is decoupled from the downstream vignette wire-up)

🤖 Generated with Claude Code

NewGraphEnvironment and others added 8 commits May 5, 2026 09:05
Lit-review temperature-departure methodology + interpretation backing.
Issue 1 of 3 in the climate-departure 3-split lit reviews
(temperature, precip+drying, interpretation framing). Mirrors the
#53/#54/v0.1.7 snowpack-lit-review pattern verbatim. Phase 0 renames
existing rag scripts to noun_verb convention before new work begins.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Switch the 3 existing ragnar build/query scripts from verb_noun to
noun_verb so the scripts/ directory reads cleanly for human users
and matches the cd_* function naming convention (cd_cog_write, not
cd_write_cog):

  rag_build_snow_methodology.R    -> rag_snow_methodology_build.R
  rag_query_snow_methodology.R    -> rag_snow_methodology_query.R
  rag_build_departure_framing.R   -> rag_departure_framing_build.R

Also updates internal docstring usage lines, CLAUDE.md script
references, and the snow-lit-review archive README how-to-reproduce
section.

Phase 0 prep for #58 — establishes the naming convention before
adding scripts/rag_temp_methodology_build.R + _query.R in Phase 3.

NEWS.md and the archive task_plan/findings/progress files left
as-is — historical records of what shipped at v0.1.7 stay named
for that version.

Refs #58.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Companion to 521084f (rename commit). Updates internal references
that point at the old script names — lost from 521084f because the
preceding heredoc-style commit message failed to parse and the
`git add -A` never ran:

- 3 renamed scripts: docstring header + Usage line
- CLAUDE.md (lines 107-108): script-reference bullets + naming-
  convention note
- planning/archive/2026-05-issue-53-snow-lit-review/README.md:
  how-to-reproduce section
- planning/active/{task_plan,findings,progress}.md: PWF tracking

Refs #58.
…cross-rag)

Targeted web search confirmed DOIs + OA paths for 10 new papers
covering: DTR / day-night asymmetry methodology (Karl 93, Easterling
97, Vose 05), Canadian / BC temperature trends (Vincent 18),
elevation-dependent warming (Pepin 15, Rangwala & Miller 12),
BC downscaling (Wang 12 ClimateWNA), climate-fish bridge (Mantua 10),
and salmonid thermal envelope (Eaton & Scheller 96, Richter & Kolmes
05).

Deep screen of the existing 19 items in NewGraphEnvironment/climate
identified 7 reuse-relevant items (Mora 13, Hersbach 20, Munoz-Sabater
21, Isaak 17 NorWeST, Dierauer 20, Warkentin 22, Moore 22) — these get
linked by item key without re-adding. Two cross-rag references from
the snow methodology rag (Najafi 17, Yue & Wang 02) cover trend-test
methodology and BC attribution without needing duplicate entries.

PDF acquisition strategy: 4 papers OA-fetchable directly (UNL
DigitalCommons, AMS post-embargo, NOAA-hosted PDFs, Canada.gov
T&F landing), 6 paywalled and flagged for user ResearchGate download
in Phase 2.

12-topic coverage matrix in findings.md maps every #58 vignette
claim type to its primary + supporting citations, ready for the
Phase 5 cite-this-for-that map.

Refs #58.
POSTed all 10 candidate papers to NewGraphEnvironment/climate (key
8MH9LCC9) via Zotero Web API with PDFs attached via 4-step S3 upload.
CrossRef-driven metadata; tags temperature-departure-methodology +
cd-issue-58. 3 fresh PDF uploads, 7 deduped via md5 against existing
Zotero S3 storage.

PDFs sourced: 1 via curl (Wang 12 from UAlberta), 9 user-provided via
ResearchGate. Karl 93 + Richter & Kolmes 05 needed OCR (image-only
scans). All 10 in data/rag/temp_methodology_pdfs/ with text layers
verified, gitignored.

Initial run mistakenly stuffed Citation Key: <clean_key> overrides
into the extra field — caught on review since NGE convention is BBT-
auto-derived keys, not manual. PATCH'd all 10 items to clear the
override. soul#43 filed to update /lit-search + /zotero-api skills so
future runs avoid this pattern.

User action needed: restart Zotero desktop so BBT generates the
citation keys for the 10 new items (sync alone does not trigger key
generation for Web-API-created items per CLAUDE.md). Once restarted,
keys get captured into findings.md and the Phase 3 rag build script.

Phase 2 deliverable: parent itemKey + attachKey table for all 10
papers in findings.md, PDFs in local cache ready for ragnar ingestion.

Refs #58.
Adds scripts/rag_temp_methodology_build.R cloning the snow build script
with a 10-paper pdf_specs map (label + Zotero attachKey + note).
Reads PDFs out of data/rag/temp_methodology_pdfs/ — same local-cache
pattern as snow, sidesteps the "download files at sync" Zotero
desktop dependency.

Runs in ~28 s on Ollama nomic-embed-text:
  Found 10 / 10 PDFs
  Chunks: 677
  Sources: 10

Sanity-tested retrieval on a DTR query — returns Karl 1993 abstract +
DTR variable construction, Rangwala & Miller 2012 alpine trends, and
Vincent 2018 nighttime asymmetry. All on-topic for the temperature
methodology citation backbone.

The Zotero attachKeys hardcoded in pdf_specs serve downstream
auditing; the BBT-auto-derived citation keys (which is what the
vignette will cite via [@key] markers) get captured in findings.md
once the user restarts Zotero desktop and BBT generates keys for the
10 Web-API-created items.

Next: Phase 4 — write rag_temp_methodology_query.R + mine the store
for methodology quotes across 8 topics.

Refs #58.
Phase 4 — adds scripts/rag_temp_methodology_query.R (clone of the
snow query script) running 24 queries across 8 topics and capturing
top-5 chunks each to planning/active/temp_methodology_quotes.md
(637 lines). Topics cover DTR asymmetry, Tmax/Tmin globe trends,
Canadian/BC temperature trends, BC downscaling, elevation-dependent
warming, climate-stream-temp bridge, salmonid thermal envelope, and
trend methodology cross-checks.

Phase 5 — synthesis in findings.md: per-topic methodology quotes
selected from the rag retrieval, cross-cutting methodology section
(baseline window, trend test cross-check vs Vincent 18's
AC-iterative procedure, ERA5-Land 2m temperature validation gap),
4 documented deviations (UTC-day tmax/tmin per issue #37, raw MK
vs AC-iterative, no direct ERA5-Land 2m T validation paper for BC,
regional DTR ratio may differ from global), and an 18-row
"cite this for that" menu mapping vignette claim types to citation
keys.

Philosophy preface added to findings.md per user feedback
(2026-05-05): this is a library, not a prescription. The downstream
vignette branch picks sparingly — citations only for authorities on
findings actually visible in the AOI's graphs/tables, plain language,
spelled-out acronyms, no over-decoration. Captured in memory as
feedback_vignette_citations_sparse.md so future runs don't bloat
findings.md with vignette-write expectations.

Phase 5 also documented:
- 7 existing items in NewGraphEnvironment/climate that are
  reuse-relevant (no re-add): mora_etal2013, hersbach_etal2020,
  munoz-sabater_etal2021, isaak_etal2017, dierauer_etal2020,
  warkentin_etal2022, moore_schindler2022
- 2 cross-rag references from data/rag/snow_methodology.duckdb that
  cover trend-test methodology and BC attribution without duplicate
  Zotero entries: najafi_etal2017, yue_wang2002

Next: Phase 6 — code-check, PR, archive. User action pending: Zotero
desktop restart so BBT generates citation keys for the 10 new items;
keys captured into findings.md before PR.

Refs #58.
After Zotero restart and BBT plugin update from 8.0.25 to 9.x
(Z8/9 line; Z9 had auto-disabled BBT 8.x as incompatible), all 10
new climate-collection items now have BBT-auto-derived keys
following the firstauthor_etalYEAR + TitlePrefix convention.

Pepin 2015 required a creators-PATCH first: CrossRef returned only
the corporate author "Mountain Research Initiative EDW Working
Group" with no individuals, so BBT initially fell back to a
title-derived key. Added the 21 individual working-group members
listed in the paper roster (N. Pepin first as corresponding
author). With BBT alive again, the key regenerated to
pepin_etal2015Elevationdependentwarming.

Final 10 BBT keys captured in findings.md Phase 2 table:
  karl_etal1993NewPerspective
  easterling_etal1997MaximumMinimum
  vose_etal2005Maximumminimum
  vincent_etal2018ChangesCanadas
  pepin_etal2015Elevationdependentwarming
  rangwala_miller2012Climatechange
  wang_etal2012ClimateWNAHighb
  mantua_etal2010Climatechange
  eaton_scheller1996Effectsclimate
  richter_kolmes2005MaximumTemperaturea

cite-this-for-that map updated to use full BBT keys (replacing
the local-label placeholders) so the downstream vignette branch
can copy [@key] markers verbatim. Also captures keys for the 7
existing reuse-relevant climate-collection items + 2 cross-rag
references (najafi_etal2017AttributionObserved,
yue_wang2002Applicabilityprewhitening) for completeness.

Refs #58.
@NewGraphEnvironment NewGraphEnvironment merged commit fa8d74a into main May 5, 2026
1 check passed
@NewGraphEnvironment NewGraphEnvironment deleted the 58-temperature-lit-review branch May 5, 2026 18:11
NewGraphEnvironment added a commit that referenced this pull request May 5, 2026
…/3) (#64)

* Initialize PWF baseline for #63

* Phase 1: interpretation framing candidate list (4 new + 6 reuse + 4 cross-rag)

* Phases 2+3+4+5: framing rag store + query + synthesis (with auto-restart)

Phase 2 — POSTed 4 candidates to NewGraphEnvironment/climate
(8MH9LCC9) via Web API with PDFs attached via S3. CrossRef-driven
metadata; tags interpretation-framing-methodology + cd-issue-63.
2 fresh PDF uploads (Arguez & Vose, Hawkins & Sutton), 2 md5-
deduped (Hansen 2012, Livezey 2007). Auto-restart fired and all 4
BBT keys captured cleanly:
  arguez_vose2011DefinitionStandard
  livezey_etal2007EstimationExtrapolation
  hawkins_sutton2012Timeemergence
  hansen_etal2012Perceptionclimate

Phase 3 — adds scripts/rag_interpretation_framing_build.R cloning
the precip+drying build script with a 4-paper pdf_specs map. Runs
in ~10 s on Ollama nomic-embed-text:
  Found 4 / 4 PDFs
  Chunks: 291
  Sources: 4

Phase 4 — adds scripts/rag_interpretation_framing_query.R running
16 queries across 6 topics (narrower than #58/#61's 24 queries
since framing topic surface is smaller). Captures top-5 chunks each
to planning/active/interpretation_framing_quotes.md (373 lines).
Topics: baseline window methodology, normals when trends exist,
time of emergence, cumulative-impact / loaded dice, shifting
baseline climate, departure from recent variability.

Phase 5 — synthesis in findings.md: per-topic methodology quotes
selected from the rag retrieval, cross-cutting methodology section
(Hansen 2012's choice of 1951-1980 base period validates cd's
choice for cumulative-impact reporting — strongest direct precedent
across all three lit reviews), 3 documented deviations (1951-1980
vs WMO 1961-1990 baseline, no AC correction, ToE not quantified
per-AOI), and an 11-row cite-this-for-that menu mapping vignette
framing claim types to BBT-auto-derived citation keys.

3-split scoreboard added to findings.md — pointer to the four
findings.md files (#53/#54 snow, #58/#60 temp, #61/#62 precip+
drying, this) for the downstream vignette wire-up branch.

Refs #63.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Lit-review temperature-departure methodology + interpretation backing

1 participant